Toward an Ethical Experiment

Randomized Controlled Trials (RCTs) enroll hundreds of millions of subjects and involve many human lives. To improve subjects’ welfare, I propose an alternative design of RCTs that I call Experiment-as-Market (EXAM). EXAM Pareto optimally randomly assigns each treatment to subjects predicted to experience better treatment effects or to subjects with stronger preferences for the treatment. EXAM is also asymptotically incentive compatible for preference elicitation. Finally, EXAM unbiasedly estimates any causal effect estimable with standard RCTs. I quantify the welfare, incentive, and information properties by applying EXAM to a water cleaning experiment in Kenya (Kremer et al., 2011). Compared to standard RCTs, EXAM substantially improves subjects’ predicted well-being while reaching similar treatment effect estimates with similar precision

Experiment-as-Market: Incorporating Welfare into Randomized Controlled Trials

Randomized Controlled Trials (RCTs) enroll hundreds of millions of subjects and involve many human lives. To improve subjects’ welfare, I propose a design of RCTs that I call Experiment-as-Market (EXAM). EXAM produces a Pareto efficient allocation of treatment assignment probabilities, is asymptotically incentive compatible for preference elicitation, and unbiasedly estimates any causal effect estimable with standard RCTs. I quantify these properties by applying EXAM to a water cleaning experiment in Kenya (Kremer et al., 2011). In this empirical setting, compared to standard RCTs, EXAM substantially improves subjects’ predicted well-being while reaching similar treatment effect estimates with similar precision

(Non)Randomization: A Theory of Quasi-Experimental Evaluation of School Quality

Many centralized school admissions systems use lotteries to ration limited seats at oversubscribed schools. The resulting random assignment is used by empirical researchers to identify the effect of entering a school on outcomes like test scores. I first find that the two most popular empirical research designs may not successfully extract a random assignment of applicants to schools. When do the research designs overcome this problem? I show the following main results for a class of data-generating mechanisms containing those used in practice: One research design extracts a random assignment under a mechanism if and practically only if the mechanism is strategy-proof for schools. In contrast, the other research design does not necessarily extract a random assignment under any mechanism

(Non)Randomization: A Theory of Quasi-Experimental Evaluation of School Quality

In centralized school admissions systems, rationing at oversubscribed schools often uses lotteries in addition to preferences. This partly random assignment is used by empirical researchers to identify the effect of entering a school on outcomes like test scores. This paper formally studies if the two most popular empirical research designs successfully extract a random assignment. For a class of data-generating mechanisms containing those used in practice, I show: One research design extracts a random assignment under a mechanism if and almost only if the mechanism is strategy-proof for schools. In contrast, the other research design does not necessarily extract a random assignment under any mechanism

Promoting School Competition Through School Choice: A Market Design Approach

We study the effect of different school choice mechanisms on schools' incentives for quality improvement. To do so, we introduce the following criterion: A mechanism respects improvements of school quality if each school becomes weakly better off whenever that school becomes more preferred by students. We first show that no stable mechanism, or mechanism that is Pareto efficient for students (such as the Boston and top trading cycles mechanisms), respects improvements of school quality. Nevertheless, for large school districts, we demonstrate that any stable mechanism approximately respects improvements of school quality; by contrast, the Boston and top trading cycles mechanisms fail to do so. Thus a stable mechanism may provide better incentives for schools to improve themselves than the Boston and top trading cycles mechanisms.Matching; School Choice; School Competition; Stability; Efficiency

Algorithm is Experiment: Machine Learning, Market Design, and Policy Eligibility Rules

Algorithms produce a growing portion of decisions and recommendations both in policy and business. Such algorithmic decisions are natural experiments (conditionally quasirandomly assigned instruments) since the algorithms make decisions based only on observable input variables. We use this observation to develop a treatment-effect estimator for a class of stochastic and deterministic algorithms. Our estimator is shown to be consistent and asymptotically normal for well-defined causal effects. A key special case of our estimator is a high-dimensional regression discontinuity design. The proofs use tools from differential geometry and geometric measure theory, which may be of independent interest. The practical performance of our method is first demonstrated in a high-dimensional simulation resembling decision-making by machine learning algorithms. Our estimator has smaller mean squared errors compared to alternative estimators. We finally apply our estimator to evaluate the effect of Coronavirus Aid, Relief, and Economic Security (CARES) Act, where more than $10 billion worth of relief funding is allocated to hospitals via an algorithmic rule. The estimates suggest that the relief funding has little effect on COVID- 19-related hospital activity levels. Naive OLS and IV estimates exhibit substantial selection bias

Curse of Democracy: Evidence from 2020

Countries with more democratic political regimes experienced greater GDP loss and more deaths from Covid-19 in 2020. Using five different instrumental variable strategies, we find that democracy is a major cause of the wealth and health losses. This impact is global and is not driven by China and the US alone. A key channel for democracy’s negative impact is weaker and narrower containment policies at the beginning of the outbreak, not the speed of introducing policies

Curse of Democracy: Evidence from the 21st Century

Democracy is widely believed to contribute to economic growth and public health. However, we find that this conventional wisdom is no longer true and even reversed; democracy has persistent negative impacts on GDP growth since the beginning of this century. This finding emerges from five different instrumental variable strategies. Our analysis suggests that democracies cause slower growth through less investment, less trade, and slower value-added growth in manufacturing and services. For 2020, democracy is also found to cause more deaths from Covid-19

Efficient Counterfactual Learning from Bandit Feedback

What is the most statistically efficient way to do off-policy optimization with batch data from bandit feedback? For log data generated by contextual bandit algorithms, we consider offline estimators for the expected reward from a counterfactual policy. Our estimators are shown to have lowest variance in a wide class of estimators, achieving variance reduction relative to standard estimators. We then apply our estimators to improve advertisement design by a major advertisement company. Consistent with the theoretical result, our estimators allow us to improve on the existing bandit algorithm with more statistical confidence compared to a state-of-theart benchmark